A conventional augmented reality (AR) game system is constructed by an AR game apparatus for controlling an AR game, a head-mounted display (to be abbreviated as an HMD hereinafter) which the player of the AR game wears, and a display for presenting a subjective viewpoint video of the player to a third party other than the player.
The AR game apparatus controls the AR game, and generates information that pertains to rendering of a virtual object at the same time. The AR game apparatus generates a composite video as a video obtained by compositing a video of a virtual object generated using the information that pertains to rendering of the virtual object, and a video of a real space at the subjective viewpoint of the player, which is sensed by a camera attached to or built in the HMD.
The composite video is displayed on the display screen of the HMD. The player plays the AR game while reviewing this composite video. In addition to the player, the video at the subjective viewpoint of the player as this composite video is presented to a third party who watches this AR game in addition to the player by displaying that video on a display prepared for a third party other than the player.
Some TV programs use virtual studios in each of which a program stage set is built by virtual objects. In this virtual studio, a program video is generated by compositing an actually sensed video of characters of the program sensed by a set camera, and a video of the virtual studio seen at the position and posture of that camera.
In the conventional AR (augmented reality) game system, a third party (watcher) other than the player can only watch a video generated for the player (mainly at the subjective viewpoint of the player). For this reason, the watcher cannot watch videos of the AR game at viewpoints other than the subjective viewpoint of the player, e.g., the overall view of the AR game viewed from the objective viewpoint, a video of the AR game at a desired viewpoint of the watcher, and the like and can neither recognize the overall status of the AR game nor watch the AR game from a desired viewpoint.
Also, a program video using a virtual studio is generated at the objective viewpoint of a third party other than the characters of the program, but a video viewed from the subjective viewpoint of each character of the program cannot be generated.
It is, therefore, an object of the present invention to generate a video of an AR game at the subjective viewpoint of the player and also generate the overall view of the AR game viewed from an objective viewpoint and a video of the AR game viewed from a desired viewpoint of the watcher.